Evolutionary games in the multiverse 
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Evolutionary game dynamics of two players with two strategies has been studied in great detail. These 
games have been used to model many biologically relevant scenarios, ranging from social dilemmas in 
mammals to microbial diversity Some of these games may in fact take place between a number of 
individuals and not just between two. Here, we address one-shot games with multiple players. As long as 
we have only two strategies, many results from two player games can be generalized to multiple players. 
For games with multiple players and more than two strategies, we show that statements derived for 
pairwise interactions do no longer hold. For two player games with any number of strategies there can be 
at most one isolated internal equilibrium. For any number of players d with any number of strategies n, 
there can be at most (d — l)™ -1 isolated internal equilibria. Multiplayer games show a great dynamical 
complexity that cannot be captured based on pairwise interactions. Our results hold for any game and 
can easily be applied for specific cases, e.g. public goods games or multiplayer stag hunts. 



Game theory was developed in economics to describe so- 
cial interactions, but it took the genius of John Maynard 
Smith and George Price to transfer this idea to biology and 
develop Evolutionary Game Theory [1-3]. Numerous books 
and articles have been written since. Typically, they begin 
with an introduction about evolutionary game theory and 
go on to describe the Prisoners Dilemma, which is one of 
the most intriguing games because rational individual deci- 
sions lead to a deviation from the social optimum. In an 
evolutionary setting, the average welfare of the population 
decreases, since defection is selected over cooperation. How 
can a strategy spread that decreases the fitness of an actor, 
but increases the fitness of its interaction partner? Vari- 
ous ways to solve such social dilemmas have been proposed 
[4j [5] . In the multiplayer version of the Prisoners Dilemma, 
the Public Goods Game, a number of players take part by 
contributing into a common pot. Interest is added to it and 
then the amount is split equally amongst all, regardless of 
whether they have contributed or not. Since only a fraction 
of one's own investment goes back to each player, there is no 
incentive to deposit anything. Instead, it is tempting only to 
take the profits of the investments of others. This scenario 
has been analyzed in a variety of contexts [6j [7] . The evo- 
lutionary dynamics of more general multiplayer games has 
received considerably less attention and we can guess why 
from the way Hamilton put it, "The theory of many-person 
games may seem to stand to that of two-person games in 
the relation of sea-sickness to a headache" (8]. Only re- 
cently, this topic has attracted renewed interest [9Hl4j. 

As shown by Broom et al. (9], the most general form of 
multiplayer games, a straightforward generalization of the 
payoff matrix concept, leads to a significant increase in the 
complexity of the evolutionary dynamics. While the evolu- 
tion of cooperation is an important and illustrative example, 
typically it does not lead to very complex dynamics. On the 
other hand, intuitive explanations for more general games 
are less straightforward, but only they illustrate the full dy- 



namical complexity of multiplayer games [9j. 

To approach this complexity, we discuss evolutionary dy- 
namics in finite as well as infinite populations. For finite 
populations, we base our analysis on a variant of the Moran 
process [15], but under weak selection, our approach is valid 
for a much wider range of evolutionary processes, see next 
section. We begin by recalling the well studied two player 
two strategies scenario. Then, we increase the number of 
players which results in a change in the dynamics and some 
basic properties of the games. For infinitely large popula- 
tions, we explore the dynamics of multiplayer games with 
multiple strategies and illustrate that this new domain is 
very different as compared to the two player situation (see 
also [9]). We provide some general results for these mul- 
tiplayer games with multiple strategies. The two strategy 
case and the two player scenario are then a special case, a 
small part of a bigger and more complex multiverse. 



I. MODEL AND RESULTS 

Two player games with two strategies have been studied 
in detail, under different dynamics and for infinite as well 
as for finite population sizes. Typically, two players meet, 
interact and obtain a payoff. The payoff is then the basis for 
their reproductive success and hence for the change in the 
composition of the population (5J. This framework can be 
used for biological systems, where strategies spread by ge- 
netic reproduction, and for social systems, where strategies 
spread by cultural imitation. 

Consider two strategies, A and B. We define the payoffs 
by oti where a is the strategy of the focal individual and the 
subscript i is the number of remaining players playing A. 
For example, when an A strategist meets another person 
playing A she gets a\. She gets ao when she meets a B 
strategist. This leads to the payoff matrix 
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Some of the important properties of two player games are: 

(1) Internal equilibria. When A is the best reply to B 
(ao > bo) and B is the best reply to A (bi > ai), 
the replicator dynamics predicts a stable coexistence 
of both strategies. Similarly, when both strategies are 
best replies to themselves, there is an unstable co- 
existence equilibrium. A two player game with two 
strategies can have at most one such internal equilib- 
rium. 

(2) Comparison of strategies. In a finite population, strat- 
egy A will replace B with a higher probability than 
vice versa if Na + (N - 2)a x > (N - 2)b + Nbi. 
This result holds for the deterministic evolutionary dy- 
namics discussed by Kandori et al. [16] . for the Moran 
process and a wide range of related birth death pro- 
cesses under weak selection [151 [17] and for some 
special processes for any intensity of selection [17]. 
However, Fudenberg et al. [18] obtain a slightly dif- 
ferent result for an alternative variant of the Moran 
process under non-weak selection. For large popula- 
tions, the condition above reduces to risk dominance 
of A, a\ + ao > b\ + 6q- 

(3) Comparison to neutrality. For weak selection, the fix- 
ation probability of strategy A in a finite population is 
larger than neutral (1/N) \f (2N-l)a -\-(N-2)a 1 > 
(27V - 4)b + (N + l)6i. For a large N, this means 
that A has a higher fitness than B at frequency 1/3, 
termed as the one-third law [T9H2T] . The 1/3-law 
holds under weak selection for any process within the 
domain of Kingman's coalescence [22] . 

Often, interactions are not between two players, but be- 
tween whole groups of players. Quorum sensing, public 
transportation systems or climate preservation represent ex- 
amples for systems in which large groups of agents interact 
simultaneously. Starting with the seminal work of Gordon 
and Hardin on the tragedy of the commons [23, 24], such 
multiplayer games have been analyzed in the context of the 
evolution of cooperation [25-28], but general multiplayer in- 
teractions have received less attention, see however [9HI3]. 

We again assume there to be two strategies A and B. 
We can also maintain the same definition of the payoffs as 
ai. As there are d— 1 other individuals, excluding the focal 
player, i can range from to d — 1. We can depict the 
payoffs ai in the form 
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However, for multiplayer games an additional complication 
arises. Consider a three player game (d — 3). Let the focal 
player be playing A. As d = 3 there are d — 1 = 2 other 
players. If one of them is of type A and the other one of type 
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FIG. 1: For 2x2 games, the payoff matrix has 4 entries. 
If we increase the number of strategies, the payoff matrix 
grows in size. For example, the payoff matrix of a 3 x 3 
game has 9 entries. If we increase the number of players, the 
payoff matrix becomes higher dimensional. For example, two 
strategy games with three players are described by 2 x 2 x 2 
payoff structures with 8 entries. In general, a d player game 
with n strategies is decribed by n d payoff values. 



B, there can be the combinations AAB or ABA. Do these 
two structures give the same payoffs? Or, in a more gen- 
eral sense, does the order of players matter? If order does 
matter, the payoffs are in a d-dimensional discrete space as 
illustrated by Fig. [l] 

There are numerous examples where the order of the play- 
ers is very important. In a game of soccer, it is necessary 
to have a player specialized as the goal keeper in the team. 
But it is also important that the goal keeper is at the goal 
and not acting as a centre-forward. A biological example 
has been studied by Stander in the Etosha National Park 
|29| . The lionesses hunt in packs and employ the flush and 
ambush technique. Some lie in ambush while others flush 
out the prey from the flanks and drive them towards the 
ones waiting in ambush. This technique needs more than 
two players to be successful. Some lionesses always display 
a particular position to be a preferred one (right flank, left 
flank or ambush). The success rate is higher if the lionesses 
are in their preferred positions. Thus, the ordering of players 
matters here. 

To address situations in which the order of player matters, 
we have to make use of a tensor notation for writing down 
the payoffs which offers the flexibility to include higher di- 
mensions of the payoff matrix. Consider a tensor f3 with d 
indices defined as follows f3i ,i 1: i 2 ,i 3 ,....i d - 1 , where the first 
index denotes the focal player's strategy. Each of the in- 
dices represents the strategy of the player in the position 
denoted by its subscript. The index i can represent any of 
the n strategies. Thus the total number of entries will be 
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n d . This structure is the multiplayer equivalent to a payoff 
matrix, see [9] and Fig. [I] Consider for example a game 
with three players and two strategies (A and B). If the 
order of players matters, then the payoff values for strategy 
A are represented by Paaa,Paab,Paba and (3 A bb- This 
increase in complexity is handled by the tensor notation but 
not reflected in the tabular notation Eq. §2§. But as long 
as interaction groups are formed at random, we can trans- 
form the payoffs such that they can be written in the form 
of Eq. ([2]), see Supporting Text. In this case, the payoffs 
are weighted by their occurrence to calculate the average 
payoffs. For example in our three player games, a\ has to 
be counted twice (corresponding to Paab and Paba)- If 
we would consider evolutionary games in structured popu- 
lations instead of random interaction group formation, then 
the argument breaks down and the tensor notation cannot 
be reduced. 

In case of d player games with two strategies we can then 
write the average payoff tta obtained by strategy A in an 
infinite population as tta = Ylt=o i^ 1 )^^ ~ x) d ~ 1 ~ k ak, 
where x is the fraction of A players. An equivalent equa- 
tion holds for the average payoff ttb of strategy B. The 
replicator equation of a 2-player game is given by (30] 

x = x(l - x)(ir A - 7Tb). (3) 

Obviously, there are two trivial fixed points when the whole 
population consists of A {x = 1) or of B (x = 0). In 
d player games, both tta and 7r# can be polynomials of 
maximum degree d — 1, see Supporting Text. This implies 
that the replicator equation can have up to d — 1 interior 
fixed points. In the two strategy case, these points can be 
either stable or unstable. The maximum number of stable 
interior fixed points possible are d/2 for even d and (d— 1)/2 
for odd d, see also [10] or [9], where it is shown that all 
these scenarios are also attainable. For d = 2, tta and ttb 
are polynomials of degree 1, hence there can be at most one 
internal equilibrium, which is either unstable (coordination 
games) or stable (coexistence games). For d = 3, there 
can also be a second interior fixed point. If one of them is 
stable, the other one must be unstable. This can lead to a 
situation in which A is advantageous when rare (the trivial 
fixed point x = is unstable), becomes disadvantageous at 
intermediate frequencies, but advantageous again for high 
frequencies, as in multiplayer stag hunts [TT] . 

For a d player game to have d—1 interior fixed points, 
the quantities — bk and a^+i — bk+i must have different 
signs for all k. However, this condition is necessary (because 
the direction of selection can only change d — 1 times if the 
payoff difference a& — bk changes sign d—1 times), but 
not sufficient, see Supporting Text. Pacheco et al. have 
studied public goods games in which a threshold frequency 
of cooperators is necessary for producing any public good 
[TTI [T2] . The payoff difference changes sign twice at this 
threshold value and hence there can be at most two internal 
equilibria. 

A d player game has a single internal equilibrium if a^ — bk 
has a different sign than a^+i — bk+i for a single value 



of k\ In this case, A individuals are disadvantageous at 
low frequency and advantageous at high frequency (or vice 
versa). If — bk changes sign only once, then the direction 
of selection can change at most once. Thus, this condition 
is sufficient in infinite populations. 

Now we deviate from the replicator dynamics, where the 
average payoff of a strategy is equated to reproductive fit- 
ness, and turn our attention to finite populations. In this 
case, the sampling for tta and ttb is no longer binomial, 
but hypergeometric, see Supporting Text. In finite popu- 
lations, the intensity of selection measures how important 
the payoff from the game is for the reproductive fitness. 
We take fitness as an exponential function of the payoff, 
Ja = ex.p(-\-WTTA) for A players and Jb = exp(+iu7re) for 
B players [31] . If w ^> 1, selection is strong and the average 
payoffs dictate the outcome of the game, whereas if w <C 1 
then selection is weak and the payoffs have only marginal ef- 
fect on the game. This choice of fitness recovers the results 
of the usual Moran process introduced by Nowak et al. [15] 
and simplifies the analytical calculations significantly under 
strong selection [31j . However, for non-weak selection other 
payoff to fitness mappings lead to slightly different results 
[15] . We employ the Moran process to model the game, 
but our results hold for any birth-death process in which 
the ratio of transition probabilities can be approximated un- 
der weak selection by a term linear in the payoff difference 
in addition to the neutral result. In the Moran process, an 
individual is selected for reproduction at random, but pro- 
portional to its fitness. The individual produces identical 
offspring. Another individual is chosen at random for death. 
With this approach we can address the basic properties of d 
player games with 2 strategies generalizing quantities from 
2x2 games. 

Does A replace B with a higher probability than vice 
versa? Comparing the fixation probabilities of a single A or 
B individual, pA and we find that pA > Pb is equivalent 
to 

d-l d-1 
A;=0 A:=0 

see Supporting Text. For d = 2, we recover the risk dom- 
inance from above. For large N, the condition reduces to 

m 

d-l d-l 

^2&k> y^frfc- (5) 

k=0 k=0 

These two conditions are valid for any intensity of selection 
in our variant of the Moran process. 

The one third law for 2-player games is not valid for higher 
number of players, see Supporting Text. Instead, the condi- 
tion we obtain for the payoff entries is not directly related to 
the internal equilibrium points (as opposed to the two player 
case, which makes the one third law special). For weak se- 
lection, we show in the Supporting text that pA > 1/N is 



4 



equivalent to 

d-i 
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(6) 
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For large population size this reduces to [13] 



d-i 



y^(d — k)dk > ^^(d — k)bk 



(7) 



k=0 



which is the one-third law from above for d = 2. Inequality 
([?]) means that the initial phase of invasion is of most im- 
portance: The factor d — k decreases linearly with k and the 
payoff values with small indices k are more important than 
the payoff values with larger indices. Thus, the payoffs rel- 
evant for small mutant frequencies determine whether the 
condition is fulfilled. In other words, the initial invasion is 
crucial to obtain a fixation probability larger than 1/N. 

In general, the conditions (5| and Q are independent 
of each other. When Eq. ([5) is satisfied and Eq. ([7]) is 
not satisfied then both fixation probabilities are less than 
neutral (1/N). But when Eq. ^ is not satisfied and Eq. 
(IT]) is satisfied then both pA and pB are larger than neutral 
(1/N). This scenario is impossible for two player games. 

Let us now turn to multiplayer games with multiple strate- 
gies. As illustrated in Fig. [I] the payoff matrix of a two 
player game increases in size when more strategies are 
added. If more players are added, the dimensionality in- 
creases. Now we address the evolutionary dynamics of such 
games. Again we assume that interaction groups are formed 
at random, such that only the number of players with a 
certain strategy - but not their arrangement - matters. 
The replicator dynamics of a d player game with n possible 
strategies can be written as a system of n — 1 differential 
equations: 



X j (7T j - (it)) 



(8) 



where x 3 - is the frequency of strategy j, ttj is the fitness 
of the strategy j and (tt) = YT 3 =\ X 3^3 ' s tne average fit- 
ness. The evolution of this system can be studied on a 
simplex with n vertices, S n . The simplex S n is defined by 
the set of all the frequencies which follow the normalisation 
Y^j=i x j — 1- The fi^d points of this system are given 
by the combination of frequencies of the strategies which 
satisfy, 7Ti = • • • = 7r n . The vertices of the simplex where 
xj is either equal to 1 or are trivial fixed points. In addi- 
tion, there can be e.g. fixed points on the edges or the faces 
of the simplex. We speak of fixed points in the interior of 
the simplex when all payoffs are identical at a point where 
all frequencies are nonzero, Xj > for all j. The internal 
equilibria are of special interest, because they may represent 
points of stable biodiversity. For example, three strains of 
Escherichia coli competing for resources have been stud- 
ied [32] [33] . K is a killer strain which produces a toxin 
harmful to S, R does not produce toxin, but is resistant to 
the toxin of K. The sensitive strain S is affected by the 
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FIG. 2: Evolutionary dynamics in a simplex with the max- 
imum number of internal equilibria for d = 4 players and 
n — 3 strategies as given by (d — l) n_1 = 9. On the dotted 
cubic curve, we have tti = 7T3. On the full cubic curve, we 
have 7T2 = 7T3 . When both lines intersect in the interior of the 
simplex, we have an internal equilibrium. 



toxin of K. These three strains are engaged in a kind of 
rock-paper-scissors game. K kills S. S reproduces faster 
than R, not paying the cost for resistance. R is superior 
to K being immune to its toxin. The precise nature of in- 
teractions determines whether biodiversity is maintained in 
an unstructured population [30, 34] . In our context this is 
reflected by the existence of an isolated internal fixed point. 

Here, we ask the more general question whether there 
are internal equilibria in d player games with n strategies. 
If so, then how many internal equilibria are possible? It has 
been shown that for a two player game with any number of 
strategies n there can be at most one isolated internal equi- 
librium [30] [35] . In the Supporting Text, we demonstrate 
that the maximum number of internal equilibria in d players 
with n strategies is 



(d-l) n ~ 



(9) 



The maximum possible number of internal equilibria in- 
creases as a polynomial in the number of players, but expo- 
nentially in the number of strategies. For example for d = 4 
and n = 3 the maximum number of internal equilibria is 9, 
see Fig. [2] 

Note that for d = 2 we recover the well known unique 
equilibrium. For n = 2, we recover the maximum of d—1 in- 
ternal equilibria, see above. Of course, not all of these equi- 
libria are stable. Broom et al. have shown which patterns of 
stability are attainable for general 3-player 3-strategy games 

m 

This illustrates that many different states of biodiversity 
are possible in multiplayer games, whereas in two player 
games, only a single one is possible. This is a crucial point 
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FIG. 3: The probabilities of observing the different number 
of internal equilibria, to (d — l) n_1 as the system gets more 
complex in the number of strategies n and the number of play- 
ers d. Random games are generated by choosing the payoff 
entries a^, bk, • • • from a uniform distribution. If we consider 
that the order does matter and generate the random games 
based on the entries of a payoff structure with n d entries, then 
the probability of observing a particular number of equilibria 
is only slightly lower (averages over 10 8 different games with 
uniformly chosen payoff entries ajt, bk, ■ ■ •)• 



when one attempts to address the question of biodiversity 
with evolutionary game theory. In the previous example the 
studies dealing with E. coll consider the system as a d = 2 
player game with three strategies. Do we really know that 
d = 2? If strains are to be engineered to stably coexist, then 
multiple interactions (d > 2) would open up the possibility 
of multiple internal fixed points instead of the single one for 
d = 2. 

If we choose a game at random, what is the probability 
that the game has a certain number of internal equilibria? 
To this end, we take the following approach: We generate 
many random payoff structures in which all payoff entries are 
uniformly distributed random numbers [36]. For each payoff 
structure, we compute the number of internal equilibria. It 
turns out that games with many internal equilibria are the 
exception rather than the rule. For example, the probability 
to see 2 or more internal equilibria in a game with 4 players 
and 3 strategies is « 24%. The probability that a randomly 
chosen game has the maximum possible number of equilibria 
decreases with increasing number of players and number of 
strategies, see Fig. [3] 

Also the probability of having a single equilibrium reduces. 
Instead we obtain several internal equilibria in the case of 
more than two players. For two player games, the proba- 
bility to see an internal equilibrium at all decreases roughly 



exponentially with the number of strategies. This poses an 
additional difficulty in coordinating in multiplayer games, 
because several different solutions may be possible that look 
quit similar at first sight. 

II. DISCUSSION 

Multiplayer games with multiple strategies is what we 
find all around. We interact with innumerable people at 
the same time, directly or indirectly. Some interactions 
may be pairwise, but others are not. In real life, we may 
typically be engaged in many person games instead of a 
disjoined collection of two person games [8j. The evolu- 
tion and maintenance of cooperation, problems pertaining 
from group hunting to deteriorating climate, all are fields 
for a multiple number of players [28j [29j [37] [38] . They can 
have different interests and hence use different strategies. 
There are other cases like the maintenance of biodiversity 
where multiplayer interactions may lead to a much richer 
spectrum for biodiversity than the commonly analyzed two 
player interactions. The presence of multiple stable states 
also contributes to the intricate dynamics observed in main- 
tenance of biodiversity [39] . Multiplayer games may help to 
improve our understanding of such systems. The problem 
of handling multiple equilibria is not just limited to biologi- 
cal games but it also appears in economics [40] [41]. Many 
insights can be obtained by studying two player games, but 
it blurs the complexity of multiplayer interactions. Here, we 
have derived some basic rules which apply to multiplayer 
games with two strategies for finite as well as infinite pop- 
ulations and discussed the number of internal equilibria in 
d player game with n strategies, which determine how the 
dynamics proceeds. 

This theory can be applied to all kinds of games with 
any number of players and strategies and can thus be easily 
applied to public goods games, multiplayer stag hunts or 
multiplayer snowdrift games. We believe that this opens 
up new avenues where we can get analytical description 
of situations which are thought to be very complex and 
further discussions on these issues will prove to be fruitful 
due to the intrinsic importance of multiplayer interactions. 
We conclude this approach by quoting Hamilton again, "A 
healthy society should feel sea-sick when confronted with 
the endless internal instabilities of the 'solutions', 'coalition 
sets', etc., which the theory of many-person games has had 
to describe." [8]. 
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